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Abstract 

Multimodal interaction methods applied to learning environments of the English language will be a line for future research from 
the use of adapted mobile phones or PDAs. Today's mobile devices allow access and data entry in a synchronized manner 
through different channels. At the academic level we made the first analysis of English language learning on a multimodal 
experimental platform. The research will evaluate the impact of college students use for future online applications aimed at 
improving language skills through self-learning. 
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1. Introduction 

Computer Assisted language learning, CALL is suffering a important changes in computer integration in the 
English exams on-line researchs (Kern & Warschauer, 2000). This adaptation are related to the type of learning 
content and the method of development of the tests on-line (Marty, 1981) (Garcia Laborda, 2008). In Spain, the 
projects development named PAULEX Universitas and OPEN-PAU, (Garcia Laborda and Litzler, 2011), evaluate 
the development of University Entrace Examination on-line test access in web experimental environments extended 
to new environments for mobile devices (Gimenez-Lopez et al, 2011). 

In recent years there have been experiences of content and navigation adaptations on mobile devices in on-line 
English tests that have allowed assessing the potential of the web mobile (Kay, 1999), (Gimenez-Lopez et al., 2009). 

The concept of the Internet for all, requires a computer developers who create new approaches to the 
development of digital environments that allow users to work simultaneously or synchronized with the different 
methods of interaction (Last, 1989), (Matthews, 1994). 
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2. La interaction multimodal. 

Multimodal interaction is a process in which devices and users are able to make a joint interaction-whether 
auditory, visual, tactile or gestural, from anywhere, anytime accessible way (W3C, 2002) 

The user through multimodal interaction can determine the mode or modes of interaction that you use to access 
information through different types of input such as keyboard, mouse, stylus, touch screens, voice, etc ... on a 
specially developed interface for it. 

In fact this method of interaction is used in human communication in a conventional way because the way we 
communicate not only provides a single channel. For example, in verbal communication simultaneously all the 
insights gained from the sight, sound, gestures, etc. .. gives us an idea of the situation, the context in which to speak, 
recognize the caller and respond to the message received. 

The multimodality applied to digital communication environments is a field of research which has worked from 
the 70’s, but computer technology was not sufficiently prepared for it. The studies carried out during the 90’s 
focused on evaluating the effective use of multimodal input devices that would improve digital communication in 
the search for information through a natural navigation (Oviatt, 1999) (Bangalore and Johnston, 2000). Also, it has 
been developed researchs about the user preference's with certain devices against others (Oviatt, 1996) (Nishimoto 
et al. 1995), (Hauptmann, 1989). The verification of the effectiveness of using multimodal interface environment 
from user point of view showed that it is now possible with the development of new devices and methods of 
interaction (Zander et al, 2010), (Desney and Nijholt, 2010). 

In the area of the CALL, multimodal interfaces have been used to promote reading skills (Alwan et al. 2007), 
although there is no specific research project to evaluate its use in the entrance exam to the University or the test 
English assessment. Neither has shown the impact of computerized testing application in mobile devices (Norbrook 
and Scott, 2003). 

It has now made a new proposal that takes into account the creation of a multimodal application to be for English 
proficiency exams on-line based on the current design of selective testing access to the University. (Magal-Royo et 
al, 2011). 

3. MM-PAU platform. 

The experimental environment called ’’multimodal interface for testing access to the university in the English 
language,” MM-PAU, developed by the Research Center in Graphics Technology at the Polytechnic University of 
Valencia applied the multimodality criteria in the user configuration. (Figure 1). 

The English assessment exam for university entrance is divided into four sections. These are: Reading 
comprenhension, composition, grammar and oral and it was the basis for creating an online interactive exam on 
ubiquitous devices. Starting from the constraints of the adaptation of digital content from the text contents 
conventional entrance exams, (Gimenez-Lopez et al., 2011), we opted for the establishment of a set of criteria that 
navigability directly affect the selection of two multimodal communication channels: the hand touch-speech 
interaction and the voice-sound interaction because these two types of interaction can be implemented in most 
ubiquitous devices such as Smartphones. 

For general validation and data flow analysis was performed general interactive application that allowed experts 
to debug the problems arising from the final adaptation of content in the end devices. Subsequently, the result was 
verified using emulators. Finally created an interactive versions based in Android was developed in three basic 
devices: HCT Desire , Samsung Galaxy an a conventional tablet PC (Figure 2). 
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Figure 1. Data flow of MM-PAU. 


The user verification was performed with the conventional design of a survey with engineering students from the 
Polytechnic University of Valencia, using a emulation versions on computer and on the devices. The survey 
established the relationship between prior knowledge of the user with this type of device and ease to use of the final 
application with the general satisfaction level. 

The results have been satisfactory to the extent that users feel comfortable during the test run and above all the 
freedom in selecting the type of interaction desired for its implementation even if it affects the overall selection 
system multimodality selected by the user and can not be changed once selected. 
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Figure 2. Functional structure of MM-PAU on Samsung Galaxy device. 


4. Conclusion. 


The concept of multimodality and may have future applications in the field of generation on-line testing of language 
skills. Validation of this type of specific task-oriented applications has permitted to analized the possibilities that 
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will be used in the field of CALL and allowing the user to access content and exams anywhere, anytime, and 
configure their own work environment. 
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